AD-A157  957  TRANSFORMING  CENSORED  SAMPLES  AND  TESTING  FITCU) 
STANFORD  UNIV  CA  DEPT  OF  STATISTICS 
F  J  O'REILLV  ET  AL.  22  JUL  85  TR-261  N00014-76-C-0475 
UNCLASSIFIED  F/G  12/1 


AD- A 157  957 


TRANSFORMING  CENSORED  SAMPLES  AND  TESTING  FIT 


BY 

F.  J.  O'REILLY  and  M.  A.  STEPHENS 


TECHNICAL  REPORT  NO.  361 
JULY  23,  1985 


Prepared  Under  Contract 
N00014-76-C-0475  (NR-042-267) 

For  the  Office  of  Naval  Research 

Herbert  Solomon,  Project  Director 

Reproduction  in  Whole  or  in  Part  is  Permitted 
for  any  purpose  of  the  United  States  Government 


Approved  for  public  release;  distribution  unlimited. 


DEPARTMENT  OF  STATISTICS 
STANFORD  UNIVERSITY 
STANFORD,  CALIFORNIA 


Accession  For 

NTT  J  CP.AftI 
D X  t  T  TAB 


Unarm  cun s 


Just  IT  i  t','.; 


By - 

DistriVuti : n/ 


r*  CD 


1.  INTRODUCTION. 


In  this  article  we  consider  tests  of  fit  for  a  completely 
specified  continuous  distribution  F(x) ,  where  the  x-sample  is  type  I-  or 
type  II- censored.  The  censoring  may  be  done  in  various  ways:  left-  or 
ri gh t- cens oring ,  as  is  often  found  in  practice,  also  double-censoring,  or 
censoring  resulting  from  deletion  of  certain  order  statistics  from  the 
original  sample. 

The  technique  proposed  generalizes  a  method  earlier  introduced 
by  Michael  and  Schucany  (1979)  in  the  sense  that  it  applies  to  an  arbitrary 
censored  sample  and  reproduces  their  results  when  the  censoring  is  of  type  II 
and  at  one  end  only.  The  method  introduced  below  is  based  on  a  transformation 
due  to  Rosenblatt  (1952)  and  its  inverse.  All  theoretical  results  are 
derived  from  the  properties  of  these  transforms,  making  it  unnecessary  to 
analyze  each  different  kind  of  censoring  separately. 

Two  procedures  are  obtained  following  the  general  methodology 
provided  by  the  application  of  Rosenblatt's  transformations.  They  trans¬ 
form  the  original  censored  data  into  a  complete  sample  of  ordered  uniforms 
if  the  hypothesized  distribution  is  correct.  Therefore,  the  distributional 
test  for  the  censored  sample  becomes  a  test  that  a  full  sample  is  uniform. 
There  are  many  well-known  methods  of  performing  the  latter  test. 

In  Section  2,  Rosenblatt's  trans formation  is  analyzed  and  its 
inverse  introduced.  In  Section  3,  the  procedures  are  obtained  for  the 
type  II  censoring,  and  shown  to  be  based  on  simple  straight  forward  cal¬ 
culations.  Some  numerical  examples  are  given.  In  Section  4,  we  examine 
for  the  type  II  censoring,  how  well  the  transformations  work  in  terms  of 
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power,  when  followed  by  Anderson-Darling' s  A  test  for  uniformity.  ifce 

2 

transformations  discussed,  followed  by  A  are  found  to  give  good  results 
and  are  therefore  recommended  in  view  of  their  versatility. 

In  Section  5,  an  analysis  is  done  for  the  type  I  censoring  and 
the  procedures  are  obtained. 

Finally  in  Section  6,  some  general  comments  are  given. 


2.  ROSENBLATT  TRANSFORMS  AND  THEIR  INVERSES. 


It  is  well  known  that  a  test  of  fit  of  a  completely  specified 
continuous  distribution  is  equivalent  to  a  test  for  uniformity.  If  the 
original  sample  is  censored,  so  that  only  a  subset  of  the  ordered  sample 
is  available,  the  equivalent  problem  is  that  of  testing  uniformity  from  the 
corresponding  subset  of  the  uniform  ordered  sample. 

2.1.  The  Rosenblatt  transformation. 

If  a  set  of  random  variables  is  given,  consisting  of  independent 

variables  whose  marginal  distributions  are  known  and  continuous,  they  can 

be  transformed  to  a  set  of  independent  uniforms  by  mapping  each  with  its 

corresponding  distribution  function.  This  is  a  direct  application  of  the 

Probability  Integral  Transformation  (PIT) .  If,  however,  the  given 

variables  are  not  independent,  but  their  joint  distribution  is  nevertheless 

known  and  absolutely  continuous,  then  a  trans formation  due  to  Rosenblatt 

(1952)  can  be  applied  that  still  yields  independent  uniforms.  Specifically 

let  Y, ,Y_,...,Y  be  jointly  distributed  with  distribution  function 
i  &  m 

G(y, • • • »Y  )»  absolutely  continuous, 
i  «  m 

Let  G^(y^)  be  t*ie  marginal  distribution  of  Y^ 

G2/l^y2^Yl^  be  the  con<^t*onal  distribution  of  Y2  given  Yj^ 
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are  independent  variables,  uniformly  distributed  in  (0,1) .  Symbolically, 
this  trans formation  is  represented  by: 


o' ,u' ,. . . ,u' 

14  m 


Observe  that  when  applying  R  in  general,  one  could  select  any 
of  the  m!  different  orderings  of  the  integers  l,2,...,m  .  We  shall 
concentrate  on  the  R- trans  formation  applied  to  ordered  uniform  random 
variables,  to  produce  new  independent  uniforms.  In  particular,  only  two 
orderings  will  be  considered;  the  forward  ordering  given  by  the  ordered 
uniforms  and  the  backward  ordering  given  by  the  ordered  uniforms  in 
reverse . 


2.2.  Application  to  ordered  uniforms. 

Let  U,, .  >U,-.  , . . . ,U,  ,  be  a  complete  ordered  sample  of  uniforms 
(lj  (2;  (n; 

Their  joint  distribution  is  of  course  known  and  absolutely  continuous  so 
that  R  can  be  applied.  Denote  this  by  utilizing  the  forward 


ordering  of  the  indices. 


Symbolically 


U(l)'U<2)'-**'U(n) 

- > 

ordered  U(0,1)  sample 


U-  »  U_  (•>•!  U 
i  z  n 


unordered  U(0,1)  sample 


The  variables  u!  can  be  shown  to  be 
1 

0^  -  1  -  {(1  -  U<A))/(1  -  U(i_1))}n‘1+1,  i  =  l,...,n  (2.] 

where  =0  . 

If  the  original  sample  of  uniforms  is  ordered  in  reverse  and  R 
applied  to  it,  we  denote  by  R^  the  resulting  transformation  applied  to 
the  backward-ordered  uniform  sample. 

Symbolically 


U(l),U(2)"**'U(n) 


ordered  U(0,1)  sample 


4  Q* 


*t  l»  II 

u. ,  u~ , . .  • ,  u 

1  i  n 


unordered  U(0,1)  sample 


The  variables  Ih  are  given  by 


Ui  _  (U(i)/U(i+l))  '  i  *  .  ,n 


(2.2 


where  U(n+1)  -  1  * 


The  transforms  U1  and  U”  are  well  known,  see  for  example 


Malmquist  (1950) . 
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2.3.  Inverse  trams  formations . 


It  is  convenient  to  define  R^1  and  R^1  ,  the  inverses  of  R^ 
and  Rg  .  These  transform  a  complete  unordered  uniform  sample  into  a 
complete  ordered  uniform  sample. 

Let  U]/U2' "  *  *  ,Un  and  ui'U2'***'Un  ^  tw°  sets  independent 

U(0,1)  variables .  Symbolically 


u’,ul,...,u' 

±  *  n 


unordered  U(0,1)  sample 


it  can  be  shown  from  (2.1)  that 


ri=T|  U(1)'U(2) . U(n) 

%  - 

ordered  U(0,1)  sample 


i  ,  ,  1/ (n-j+1)  .  _  , 

U . . .  —  1  -  tt  (1  —  u.)  »  i  —  l>...«n 

u'  j=l  3 


(2.3) 


and  from  (3.1) , 


tl  II  It 

U.  f  U_  f  ...  f  u 
l  a  n 


Unordered  U(0,1)  sample 


U(l) ,U(2) 


ordered  U(0,1)  sample 


where 


U,..  -  it  (u")1/j  ,  i  =  1, . . . ,n  . 
U)  j=i  3 


(2.4) 


These  transforms  R^  and  R^  are  defined  because  they  "recover 
the  original  ordered  sample  if  this  was  transformed  by  Rp  or  R^ 
respectively. 


3.  APPLICATION  TO  GOODNESS  OF  FIT  WITH  TYPE  II  CENSORED  DATA. 


Transforms  T_  and  T  :  censored  samples  to  full  samples 


Suppose  a  censored  sample  of  X- values  is  given, 

X^  j  <  X^  j  ,  . ..,  X^  j  ,  and  we  want  to  test  that  they  came  from  a 
12  r 

specified  continuous  distribution  F(x) .  First,  the  corresponding  censored 

sanple  of  uniforms  is  obtained  by  U..  .  =  F(X..  .  ),  U,.  .  =  F(X.  .),  etc. 

\i^/  » ^2 

The  U  variables  have  a  joint  absolutely  continuous  distribution 
so  that  Rosenblatt's  transformation  can  be  applied  to  them  providing  a 
complete  unordered  uniform  sa-r  le  U*  ,1^, . .  •  ,U^  . 

Suppose  Rosenblatt's  trams formation  is  applied  forward,  namely 

first  mapping  j  with  its  marginal  distribution,  then  mapping  j 

1  2 

with  its  conditional  distribution  given  U..  .  ,  and  so  on,  to  give  the  set 

-1  1 

U'  ;  then  apply  R^  to  the  U'  variables  and  obtain  a  set 
*  *  * 

U(i),u(2) . u(r)  which  is  a  complete  ordered  U(0,1)  sample.  Symbolically 

let  this  resulting  transformation  be  T  : 


U(i,)/U(iJ'***'U(iJ 


U(D'U(2) . UIr) 


Subset  of  r  ordered  U(0,1) 
variables  from  n 


complete  sample  of  r 
ordered  U(0,1)  variables 


Thus  Tp  transforms  any  ordered  subset  of  uniforms  into  a  new 
complete  ordered  set  of  uniforms. 


In  a  similar  way,  one  can  define  T  ,  by  first  applying 

B 

Rosenblatt’s  transformation  to  )  ,U(i  )'*’*'U(i  )  ^ut  start*n9  with 


U..  .  ,  continuing  with  U.^  .  given  .  ,  etc.,  and  then  applying  , 

'  r1  '  r-l'  r1 

to  give  the  final  ordered  set  U*  . 

The  characterizing  properties  of  the  distribution  of  the  order 

statistics  make  both  T  and  T  doubly  invariant  characterizations  in 

the  sense  that  they  map  order  statistics  into  order  statistics  and  the  set 

U.,  U..  .  has  the  joint  distribution  of  the  corresponding  subset  of 

'  r  '  r 

ordered  uniforms  if  and  only  if  U?, ,,..., U?  .  is  distributed  as  an 

(1)  (r) 

ordered  complete  sample  of  uniforms.  In  O'Reilly  and  Stephens  (1982) ,  in 

connection  with  transformations  of  exponentials  to  uniforms,  these  concepts 

were  explored.  There  it  was  found  that  doubly  invariant  characterizations 

were  the  basis  of  procedures  with  good  power  properties.  It  is  in  this 

spirit  that  we  now  suggest  T  or  T  for  censored  data. 

F  B 

Example.  In  order  to  apply  either  T_  or  T_  in  practice  one  requires 

F  B 

knowledge  of  marginals  and  conditionals  of  uniform  order  statistics.  These 
have  nice  properties  and  so  the  application  is  quite  simple. 

Suppose  for  exanple  that  one  has  only  and  out 

of  a  sample  of  size  10  .  In  order  to  apply  T  ,  one  first  needs  the 
marginal  distribution  of  U ^  ,  which  is  a  Beta  (2,  9)  (denoted  B 2  g  (•)). 

SO  »i-B2j9(0(2))  , 

then  one  needs  the  conditional  distribution  of  given  .  Given 

Upj  ,  the  variables  U(3)  ,U(4)  • *  *  *  ,U(10)  ^e^ave  like  an  ordered  uniform  sample 

of  size  8  on  the  interval  (U^j  ,1)  ,  so  given  U ^  has  the  same 

distribution  as  the  third  order  statistic  of  a  sample  of  size  8  on  the 
interval  (U^j  * 

SO  u;  -  s3i6{(0(5)  -  o(2))/u  -  U(2))}  . 


9  *•  *  t  *  ‘  ,  '  »  *  »  '  *  *  h  .*  «  *  «  * 


Finally  we  need  the  conditional  distribution  of  given 

and  •  "This  is  the  same  as  the  conditional  distribution  of  given 

because  of  the  Markovian  property  of  order  statistics. 

Given  U._.,  the  order  statistics  U ...  ,U ,U have  the 
(o)  (6)  (7)  (10) 

same  distribution  as  a  uniform  ordered  sample  of  size  5  on  the  interval 


S°  U3  "  B2,4{(U(7)  "  U(5))/(1  "  U(5))}  * 

With  U{'U2'U3  S°  coinPutec*  one  would  finally  apply  R  ^  which  yields: 


U*  =  1  -  (1  -  U' ) 173 
(1)  v  l1 


(1  -  u^)1/3 


u^)172 


(1  -  u^)1/3 


U2)1/2  (1 


u^)171 


For  the  right-censored  sample ,  and  also  for  the  doubly-censored  sample, 
which  often  occur  in  practice,  T  and  T  are  straightforward,  as 

r  o 

follows. 

3.2.  Right-Censored  Case. 

For  U.,.  ,  U , . . . ,U .  .  out  of  n  ,  the  application  of  Rosenblatt's 
(1)  (2)  (r) 

transformation  in  a  forward  ordering  yields: 


Ul'U2'‘*"'Ur  independent  U(0,1)  variables  with 


> ,n-i+l 


11 


then  application  of  r“L  (Equation  2.3)  to  yields  the  desired 

complete  uniform  sample  of  size  r  ,  U*^ • • • • 'u*r ) * 

If  instead  we  use  T  ,  first  we  apply  Rosenblatt's  trams formation 

B 

in  a  backward  ordering  which  yields: 


iid  U(0,1)  *  where 


ai  •  fau)/au+i>>1'  1  =  1 . .  '  “d 


O"  =  B  (U.  , )  • 

r  r,n-r+l  (r) 


“1  Ml* 

Finally  R^  (Equation  2.4)  is  applied  to  which  results 

in  a  complete,  ordered  uniform  sautple  of  size  r  . 

For  this  case  T^,  does  not  require  am  evaluation  of  a  Beta 
distribution  function;  this  gives  a  computational  advantage  over  T 

B 

Note  that  for  the  right-censoring  case,  T  reconstructs  the 

B 

random  variables  given  in  Theorem  1  of  Michael  and  Schucany  (1979) . 

In  a  similar  way,  a  left  censored  sample  can  be  transformed;  in 
this  case  T  reconstructs  the  procedure  of  Michael  and  Schucany. 


3.3.  Doubly-censored  case. 

Suppose  the  available  order  statistics  from  the  uniform  sample 


of  size  n  are: 


<k+l)'  (k+2)"‘"u(k+r) 
where  1  <  k  <  n-r  . 


For  Tp  ,  first  Rosenblatt's  transformation  is  applied  starting 


with  the  marginal  of  U(k+u'  then  with  the  conditional  of  U^+2j  9iven 

n  .  and  so  on,  which  yields 
<k+l)'  • 


U1  =  Bk+l,n-k(U(k+l))  411(3 

U±  =  1  -  {(1  -  u(k+i))/(1  ~  u(jc+i-i))^  ?  i  *  2,...,r 

then  Rp1  is  applied  to  U^,...,U^  . 

For  T  ,  Rosenblatt's  transformation  yields 
£ 

•• 

°r  ”  Bk+r,n-k-r+l^U(k+r) ^ 

Ui  =  tU(k+i)/U(k+i+l)}  '  1  =  l»2,...,r-l 

—  1  n  n 

then  RQ  is  applied  to  U^,...,Ur  . 

In  this  censoring  scheme  there  seems  to  be  no  computational 

advantage  of  T  over  T  since  both  require  the  Beta  distribution. 

F  5 

3.4.  Numerical  examples. 

The  following  examples  were  taken  from  Figure  1,  p.  437  of 


Michael  and  Schucany.  The  values  of  the  actual  observations  are  interpreted 
from  the  plots. 


In  the  three  examples  that  follow  n  =  9  and  r  =  5  ,  and  the 
censoring  is  right- censoring. 

(a)  For  the  first  example  are  0.1,  0.2,  0.3,  0.4 

and  0.5  .  T  ,  as  described  in  Section  3.2,  gives  a  complete  ordered  uniform 
F  “ 

set  U*  with  values  0.1211,  0.3014,  0.4785,  0.6536  and  0.8272;  the  Anderson- 

2  2 

Darling  statistic  is  A  =  0.1884.  The  formula  for  A  ,  for  r  ordered 
values  U*  is 


A2  =  -  r  -  Z  (2i-l){loge(U*r))  +  loge  (1  -  Ufr+1_i})}]  . 


Transformation  T  gives  another  complete  uniform  set 

B 

0.1741,  0.3482,  0.5223,  0.6964  and  0.8706  with  A2  ■  0.1947. 


(b)  For  the  second  example,  U(d »• •• ,U(5)  are 

0.0206,  0.0412,  0.0618,  0.0824  and  0.1030  .  T  gives  the  set  U*  s 

F 

0.7329,  0.8210,  0.8775,  0.9231  and  0.9632,  with  A2  =  4.5746  and  T  gives  U* 

B 

0.0504,  0.1009,  0.1514,  0.2018  and  0.2523,  with  A2  =  4.1808. 


(c)  In  the  third  example,  , . . .  ,Uj5j  ar® 

0.1794,  0.3588,  0.5382,  0.7176  and  0.8970.  T_  gives  U*  s 

F 

0.0003*  0.0455,  0.7989,  0.4277  and  0.7005  with  A2  =  2.789,  and 
T&  gives  U*  ;  0.1999,  0.3999,  0.5998,  0.7998  and  0.9998,  with 

A2  *  1.3926  . 


In  example  (c) ,  there  is  a  noticeable  difference  in  applying 


T_  or  T  ;  the  p- value  for  A  after  T_  is  below  0.05  and  the  p-value 

F  B  F 

2 

for  A  after  T_  is  above  0.15  . 


4.  POWER  STUDY. 


Four  different  methods  for  testing  uniformity  from  a  type  II-censored 
sample  were  considered.  These  are  T ,  T  and  Michael  and  Schucany' s 

F  8 
2 

transformation  (MS)  each  followed  by  A  and  also  Pettitt  and  Stephens'  (1976) 

2  2 
modified  A  (called  PS) .  This  last  method  calculates  A  directly 

from  the  censored  sample,  without  any  transformation,  and  must  be  compared 

to  a  table  appropriate  to  the  censoring  being  considered.  Tables  are 

given  in  Pettitt  and  Stephens  (1976).  Other  statistics  were  shown  to  be, 

on  the  whole,  inferior  to  MS  and  PS  by  Michael  and  Schucany  (1979);  see 

their  comment  on  p.  438. 

The  alternative  distributions  considered  in  the  power  study  have 
been  used  in  previous  studies  for  uniformity  by  Stephens  (1974) , 

Quesenberry  and  Miller  (1977) ,  Michael  and  Schucany  (1979)  and  Dudewicz 
and  Vein  der  Muelen  (1981)  . 

These  alternative  non-uniforms  are  the  following; 

F^:  the  distribution  of  Z^  where  Z  ~  U(0,1). 

2 

F2:  the  distribution  of  1  -  Z  ,  where  Z  ~  U(0,1)  . 

F3:  the  distribution  of  0.5Zi  +  0.5  Z2  ,  where  Z^,  Z2  are  independently  U(0,1) 
F^s  a  mixture  of  F^  and  F2  with  equal  weights. 

F,.;  the  distribution  of  Z  -  0.5  (if  Z  >  0.5)  or  Z  +  0.5  (if  Z  <  0.5) 
where  Z  ~  F^  . 

The  different  censoring  schemes  studied  were  condensed  in  two 


separate  tables: 


(a)  The  first  censoring  scheme  is  of  a  sample  of  size  20  censored  at  both 
extremes,  thus  only 

U{k+1)  ,U(k+2) '•**,U(k+r) 
were  observed. 

The  number  r  of  available  observations  was  taken  to  be 
5,  10  or  15,  and  k  ,  which  determines  the  asymmetry  of  the  censoring  was 
varied  such  that  p  =  k/(20-r)  was  0.0,  0.2,  0.4,  0.6,  0.8  and  1.0.  lhus  p 
represents  the  fraction  of  censored  observations  at  the  left;  p  =  0  means  right 
censoring  whereas  p  =  .2  means  that  20%  of  the  censored  values  occurred 
at  the  left.  The  power  results  for  this  censoring  scheme  and  the  different 
alternatives  appear  in  table  1. 

(b)  In  the  second  censoring  scheme,  the  sample  is  censored  in  the  middle, 
thus  only  the  r  values 

U(l)  ,U{2)  '  —  ,U(«)  and  U(20-r+£+l)  '••"U(20) 
were  observed. 

The  number  r  of  available  observations  is  again  varied  as 
5,  10  and  15.  The  number  l  ,  of  available  observations  at  the  left,  was 
varied  such  that  q  =  1/ r  was  0.0,  0.2,  0.4,  0.6,  0.8  and  1.0.  The  power 
results  appear  in  table  2. 

For  both  tables,  1000  samples  were  generated  and  the  percentage 
of  times  that  the  corresponding  test  detected  the  alternative,  was  recorded. 

Ofoe  size  of  the  tests  was  in  all  cases  a  =  0.05. 


For  the  first  censoring  scheme,  that  of  censoring  at  both 
extremes,  except  for  the  cases  where  p  is  0.0  or  1.0  and  which 
correspond  to  right  and  left-censoring  respectively,  the  MS  transformation 
could  be  applied  in  two  different  ways  (see  their  comments  on  p.  439,  first 
paragraph) .  Both  possibilities  were  considered  in  table  1  with  the 
results  recorded  in  the  same  cell.  It  can  be  observed  that  both  possibilities 


lead  to  similar  power. 
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Comments  on  Table  1. 


When  p  =  0  (the  sample  is  right  censored) ,  the  results  agree 

with  those  reported  by  Michael  and  Schucany  (1979) ,  to  within  Monte  Carlo 

variation.  For  this  case,  (p  =  0)  ,  the  Pettitt-Stephens  (1976)  procedure 

dominates  under  alternative  F_  where  the  second  best  is  T  (recall  that 

2  r 

in  this  case  T_  is  equivalent  to  MS  and  also  T  has  a  computational 
B  r 

advantage  over  T_)  .  Against  F,,  T_  dominates  T  and  except  for  highly 

B  J  D  f 

censored  samples  (r  =  5) ,  also  dominates  PS. 

For  the  rest  of  the  alternatives,  still  with  p  =  0  ,  T  and 

B 

T  are  roughly  equivalent  and  dominate  PS  . 

F 

Under  left  censoring,  that  is  when  p  *  1,  procedure  PS  out¬ 
performs  the  rest  under  alternative  F  where  T  is  the  second  best 

x  B 

(here,  MS  and  coincide).  Under  F^ ,  except  for  highly  censored 

samples  (r  =  5) ,  where  PS  does  well,  T  dominates  T  and  PS  . 

F  B 

For  the  rest  of  the  alternatives,  still  with  p  =  1,  T„  and  T 

B  F 

are  again  roughly  equivalent  and  dominate  PS  . 

If  censoring  occurs  at  both  extremes,  then  the  relative 
performances  of  the  test  procedure  depend  on  the  degree  of  symmetry  of  the 
censorship,  that  is,  how  close  p  is  to  0.5. 

Against  F^,  procedure  PS  does  very  well  except  for  values  of  p 

on  the  extremes.  Against  F^  however,  PS  is  outperformed  by  MS  ,  which 

in  turn  is  outperformed  by  T  when  there  is  more  censoring  at  the  right 

F 


and  by  T  if  there  is  more  censoring  at  the  left. 


Against  F  ,  PS  and  T  behave  similarly  and  do  better  than 

MS  or  T  .  Against  F-,  PS  and  T  are  similar  and  outperform  the 

r  Z  F 

other  two. 

Against  F^  ,  there  are  no  big  discrepancies,  except  if  r  =  15 
(25%  censoring)  in  which  case  PS  does  better. 

Comments  on  Table  2. 

Consider  the  case  where  q  =  .2,  .4,  .6  or  .8  to  avoid  the 
cases  of  left-  and  right-censoring  (q  =  0  and  q  =  1) ,  already  discussed 

Agaiist  F  ,  MS  and  T?  dominate.  Against  •  MS  and 
dominate  and  against  F^ ,  PS  dominates.  Against  F^,  MS,  Tp  and  T0 
are  comparable  and  PS  behaves  poorly,  especially  with  high  censorship 
(75%). 

Against  F  ,  T  or  T  do  well  if  the  largest  amount  of 
D  F  B 

available  information  is  at  the  left  or  right  respectively.  The 
procedure  MS  behaves  symmetrically  in  this  respect  and  PS  yields  low 
power  if  censorship  is  above  60%. 
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5.  APPLICATION  TO  GOODNESS  OF  FIT  WITH  TYPE  1  CENSORED  DATA. 

In  Section  3,  the  trans formations  T„  and  Tn  were  defined  as 

F  B 

those  resulting  from  Rosenblatt' s  transformation  applied  to  the  available 
subset  of  the  order  statistics  in  a  forward  or  backward  ordering,  followed 
by  and  R^  respectively. 

For  one  sided  or  double  censoring  of  type  I,  one  can  still  find 

T  and  T  .  For  this  we  need  only  to  consider  a  conditionality  argument. 
B  F 

5.1.  Right  censored  case  (Type  I) . 

The  observations  U^j  »°(2)  • ' *  •  'U^rj  are  the  sample  values  that 
were  less  than  a  fixed  and  known  censoring  constant  t  .  Observe  that  the 
integer  r  is  random. 

Define  the  event  C  to  be  U,  .  <  t  <  U,  . 

(r)  (r+1) 

Consider  the  conditional  joint  distribution  of  • • • • ,U(r) 

given  C  .  Since  this  is  absolutely  continuous  (almost  surely) ,  one 
can  apply  Rosenblatt's  transformation  in  a  forward  or  backward  ordering. 

For  example ,  the  transformation  T  works  as  follows. 

b 

First,  the  conditional  distribution  of  given  C  is  needed 

to  map  U,  .  ,  then  the  conditional  distribution  of  U,  given  C  and 

(r)  (r-1) 

Ujrj  is  needed  to  map  U(r_i)  *  then  the  conditional  distribution  of  U(r_2) 

is  needed  to  map  U,  , ,  ,  then  the  conditional  distribution  of  U, 

(r-1)  (r-2) 

given  C  ,  U^rj  and  U(r_u  is  required  to  map  U(r_2)  *  30(1  80  on* 

The  conditional  distribution  S(»)  of  U^rj  given  C  is 


w.  ir.irj  jr. 1 


0  if  u  5  0 


Sy  (u | C)  =  <  if  u  €  (0,1 


1  if  u  2  t  . 


Similarly,  the  condi tional  distribution  of  U^r_^j  given  C  and  °(r) 


is  found  to  be 


(r-1) 


(uic,U.  .) 


0  if  u  5  0 


(»,;) 


if  u  €  (0,U(r)) 


1  if  u  >  0, 


Finally,  T0  consists  in  applying  (formula  2.4)  to  the  set 


Ul"'*'°r  9iven  bV 


°r  *  ‘V)'*1' 


Uj  "  <U(j)/U(j+l))  7  ^  "  1» • • • »r“l 


(5.1) 


The  resulting  U  are,  conditionally  on  C  ,  independent  U(0,1) 
hence  after  applying  R^1  to  these,  we  obtain  u*d  ' • • • 'u*r)  which, 
conditionally  on  C  ,  are  distributed  like  a  complete  ordered  U(0,1) 
sample. 

A  similar  derivation  yields  T  in  this  Type  I  right  censored 

F 

i  i 

case.  The  Rosenblatt  transformation  gives  U  ,...,U  where 

1  r 

0.,.  r 

u;  =  i  -  {i  -  -£4 

and  (5-2) 

o!  -  1  -  a  -  (0(i)  -  0(i.1))/(t  -  i  -  2 . r; 

then  R  ^  is  applied  to  the  to  give  the  ordered  set  U*  . 


5.2.  Double  Censored  case  (Type  I) . 


The  uncensored  observations  U„  ,  ,.,U„  U„  ,  .  are  the 

(k+1)  (k+2)  (k+r) 

sample  values  which  were  less  than  a  fixed  known  censoring  constant  t2  an 
greater  than  another  fixed  and  known  censoring  constant  t^  ,  with 
tx  <  t2  . 

Applying  Rosenblatt's  transformation  conditional 
on  the  events  C  and  D  defined  by 

C  •  lU(k«)  •=  ‘2  <  U(k«+1)  >'  D  ■  (°<k)  <  *1  <  °(k.l) 
the  following  results  are  obtained. 

For  T0,  the  . if  needed  before  R^1  is  applied  are  given  by 
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»;  -  «v«> 


tjl/ctj  -  t^)}* 


and 


(5.3) 


U. 

i 


=  {(U 


(k+i) 


tl)/(U(k+i+l) 


-  ,  i  = 


1, . . . ,r-l 


For  T  ,  the  u',...,u‘  needed  before  applying  R 1  are  given  by 
F  i  r  * 


i 

K' 

s 


r- 


i 

«— 

t 


* 

i 

» 

* 

% 

% 

* 

r. 


°r*  1  ‘  (t2  -  lW>,/(t2  -  ‘I11' 


and 


U(k+r-i+l))/(t2  “ 


U(k+r-i)^ 


(5.4) 


for  i  =  1, . . . ,r-l  . 

In  this  type  of  censoring  situation  (and  similarly  for  Type  1 

right-  or  left-censoring)  one  finishes  with  a  set  of  random  variables 

U*d . U*r)  which  conditionally  on  C  and  D  has  the  same  distribution 

as  a  complete  ordered  U(Ofl)  sample  of  size  r  .  The  Anderson-Darling 
2 

A  confuted  from  these  random  variables,  conditionally  on  C  and  D  , 

has  a  well  known  distribution,  which  in  the  upper  tail  is,  for  all 

practical  purposes ,  independent  of  r  (if  r  >  5;  see  Stephens,  1974); 

2 

hence  the  test  statistic  A  applied  after  T  or  T  in  a  type  I 

D  F 

censoring  situation  is  essentially  distributed  as  in  the  usual 
unconditional  test  for  uniformity. 


6 .  GENERAL  COMMENTS  . 


(a)  For  Type  2  censoring,  there  seems  to  be  no  overall  best  procedure 
from  the  comparison  carried  out  in  Section  4.  It  is  true,  however,  that  in 
a  practical  situation  one  knows  the  type  of  censoring  which  is  confronted 
and  if  one  suspects  the  alternative,  then  the  general  comparisons  given 
should  be  useful. 

(b)  An  appealing  property  of  the  procedures  T  and  T  is  that  they 

F  o 

indicate  explicitly  how  a  given  subset  of  an  ordered  uniform  sample  may  be 
transformed  to  a  complete  ordered  uniform  sample,  by  a  method  which  applies 
in  great  generality. 

(c)  When  defining  T  in  Section  3,  one  first  maps  the  available  order 

F 

statistics  with  Rosenblatt's  transformation  starting  with  the  smallest, 
then  the  next  to  the  smallest  and  so  on.  At  this  stage  one  has  an 
independent  U(0,1)  sample.  Any  fixed  permutation  of  this  independent 
sample  could  then  be  trans formed  with  R^  .  Theoretically  this  is  a 
valid  procedure  but  in  practice,  we  found  that  the  power  was  highest 
when  the  ordering  was  maintained.  This  empirical  result  suggested  the 
use  of  R^,1  when  the  Rosenblatt  transformation  was  used  forward  and  R^ 

when  it  was  used  with  the  backward  ordering.  In  this  way  one  maintains  the 
identity  of  the  observations.  This  agrees  with  an  observation  concerning 
the  retention  of  identity  that  appears  in  Michael  and  Schucany  (1979, 
p.  439,  in  the  first  paragraph). 

(d)  For  Type  1  censoring,  there  is  extra  information  in  the  random  number 
r  (and,  for  double  censoring,  in  k)  .  For  exanple,  with  only  right 
censoring,  a  value  r  far  from  its  expected  value  nt  indicates  lack 


of  uniformity,  even  if  the  r  values  themselves  were  evenly  distributed 
in  the  interval  (0,t).  The  need  to  make  a  test  involving  r  has  been 
discussed  by  Stephens, and  by  Michael  and  Schucany,  in  Chapters  5  and  12 
of  D'Agostino  and  Stephens  (1985) .  In  Chapter  5  it  is  suggested  that  a 
test  that  r  is  reasonable  (r  has  a  binomial  distribution  with  parameters 
n  and  t)  be  combined  with  the  test  for  uniformity.  A  two-stage  procedure 
of  this  type  has  recently  been  discussed  by  Maag  and  Dufour  (1985) .  An 
interesting  question  which  then  arises  is  how  best  to  choose  the  a- levels 
of  the  component  tests  to  give  the  desired  overall  a- level.  In 
Chapter  12  of  D'Agostino  and  Stephens  (1985) ,  Michael  and  Schucany 
point  out  that  the  Pettitt-Stephens  procedure  makes  use  of  r  ;  so  also 
does  a  statistic  based  on  the  spacings  between  the  . 

Further  work  is  needed  to  compare  these  several  methods  of 
incorporating  the  information  contained  in  r  ,  and,  for  double  censoring. 
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y'One  approach  to  testing  the  goodness-of-f it  of  a  completely  specified  continuous 
distribution  with  a  censored  sample,  is  to  transform  the  corresponding  uniform 
censored  sample  into  a  complete  uniform  sample  and  then  to  use  any  of  the  classi¬ 
cal  uniformity  tests.  Two  systematic  procedures  for  transforming  a  censored  uni¬ 
form  sample  are  proposed.  A  Monte  Carlo  power  study  was  conducted  to  analyze  the 
relative  merits  of  these  procedures  when  followed  by  the  Anderson-Darling  a£  test 
for  uniformity.  The  suggested  transformations  are  versatile  and  with  A2  ,  give 
good  power  for  most  alternatives,  - 
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